1. Field of the Invention
The present invention relates to a resolution enhancing method and apparatus for magnifying a video in at least one of a vertical direction, horizontal direction or time direction.
2. Description of the Related Art
One example of a resolution enhancing method for performing resolution enhancement on an image having a low resolution is disclosed by JP-A 2003-18398 (KOKAI). The method in the JP-A 2003-18398 (KOKAI) includes a training stage and a resolution enhancing stage. The training stage generates high-frequency component images of a training image as well as a reduced image obtained by reducing the training image in size. The training stage stores, as a look-up table, a plurality of pairs each composed of a feature vector of a block (reduced block) in the reduced image and a block (high-frequency block) in the high-frequency component image which is located at the same position as that of the reduced block (a part with the same object as the reduced block). The training stage shifts the positions of the reduced blocks to repeat the similar processing, appropriately adds the training images to repeat the foregoing processing, and then, terminates the processing.
On the other hand, the resolution enhancing stage calculates feature vectors of blocks in the input image (input blocks) as well as generates a temporary magnified image in which the input image to be enhanced in resolution is magnified. In this case, the input blocks are the same in size as those of the reduced blocks, and the feature vectors of the input blocks are calculated in the same method of the training stage.
Subsequently, the resolution enhancing stage retrieves the feature vector of the reduced block similar to the feature vector of the input block from the look-up table. The high-frequency block making a pair with the retrieved feature vector is added to the block in the temporary magnified image at the same position (temporary magnified block) to produce an output block. The temporary magnified block has the same size as that of the high-frequency block added to the temporary magnified block, and the output block is part of an output image. If the output blocks do not cover the whole of the output image, the resolution enhancing stage sifts the positions of the input blocks so as to cover the output image to repeat the same processing, and if the output blocks cover the whole thereof, the processing is terminated.
According to the method in such JP-A 2003-18398 (KOKAI), texture having become sharp as a result of the addition of the high-frequency component to the temporary magnified image for each block, the method therein can provide a sharp and high resolution image.
Reproducing, as a video, a plurality of resolution enhancing images obtained by applying the method in the JP-A 2003-18398 (KOKAI) to each frame of the video causes a time change in color at the same positions in a space direction sometimes. For example, it is presumed that the output block of a first output image in which a resolution of a t-th frame is enhanced is at the same position as that of a second output image in which a resolution of a (t+1)-th frame is enhanced in the space direction. At this point, in the training stage, the output blocks are high-frequency blocks in different two high-frequency component images, and in some cases, they are generated from positions absolutely different in space. The two high-frequency blocks have high-frequency components not continuous in the time direction as elements. Therefore, reproducing the first and second output image as the video each including the output blocks which are obtained by adding to the temporary magnified blocks causes unnatural time changes.
Such a situation, in which the blocks which are added to the same position in the space direction in succession in the time direction in the resolution enhancing stage are generated with one another from quite different positions in the training stage, occurs very frequently so far as the feature vector of the block at the same position as that of the block in the input image of the t-th frame is not perfectly identical with the feature vector of the block at the same position as that of the input image of the (t+1)-th frame. Accordingly, the reproduction of a plurality of resolution enhancing images obtained by applying the method disclosed in the JP-A 2003-18398 (KOKAI) to each frame of the video results in generating flickering with high frequency.